Matching into competitive surgical residencies: predictors of success

ABSTRACT Evidence-informed data may help students matching into competitive residency programs guide curricular activities, extracurricular activities, and residency career choices. We aimed to examine the characteristics of students applying to competitive surgical residencies and identify predictors of matching success. We identified the five lowest match rates for the surgical subspecialities listed in the 2020 National Resident Matching Program report to define a surgical residency as competitive. We analyzed a database from 115 United States medical schools regarding application data from 2017 to 2020. Multilevel logistic regression was used to determine predictors of matching. Statistical significance was set at p < 0.05.A total of 1,448 medical students submitted 25,549 applications. The five most competitive specialties included were plastic surgery (N = 172), otolaryngology (N = 342), neurological surgery (N = 163), vascular surgery (N = 52), orthopedic surgery (N = 679), and thoracic surgery (N = 40). We found that medical students with a geographical connection (adjusted OR, 1.65 [95% CI, 1.41 to 1.93]), and students who did an away rotation at the applied program (adjusted OR, 3.22 [95% CI, 2.75 to 3.78]) had statistically significantly increased odds of matching into a competitive surgical specialty. Furthermore, we found that students with a United States Medical Licensing Examination (USMLE) Step 1 score below 230 and Step 2 Clinical Knowledge (CK) score below 240 had increased odds of matching if they completed an away rotation at the applied program. Completing an away rotation and geographical connection to the institution may contribute more than academic criteria for selection into a competitive surgical residency after an interview. This finding may be due to less variation in academic criteria among this pool of high-performing medical students. Students with limited resources who apply to a competitive surgical specialty may be at a disadvantage given the financial burden of an away rotation.


Introduction
Despite efforts to increase the number of medical students matching into residency positions, matching into certain surgical specialties remains competitive. Data from the National Resident Matching Program (NRMP) reveal higher proportions of unmatched U.S. senior medical students in the specialties of otolaryngology, orthopaedic surgery, plastic surgery, and neurological surgery compared to other specialties [1]. As successful matching into postgraduate training is a primary goal of undergraduate medical education, advising and supporting medical students seeking to match into the most competitive specialties can be challenging.
Various researchers have studied the influence of certain selection criteria on medical students matching into particular residency programs [2][3][4]. For instance, Green et al. conducted a national survey of residency program directors to identify the most influential selection criteria for residency programs and found that grades in clerkships and the USMLE Step 1 score ranked highest among the selection criteria [5]. Furthermore, researchers in particular specialties, including orthopedic surgery, plastic surgery, among others, have investigated specialtyspecific factors that influence selection for residency [6,7]; however, the current literature is limited to reports of program director surveys. Moreover, there is a paucity of literature utilizing data from medical student applicants to evaluate factors that are associated with matching into competitive surgical specialties after being offered an interview.
USMLE Step 1 scores have been shown to be an integral component of the selection criteria for matching into competitive residencies [5,[8][9][10][11]. Given the recent announcement by the National Board of Medical Examiners (NBME) to change score reporting for the Step 1 board examinations to pass/fail as of January 2022, identifying which other factors are important for matching into competitive specialties will potentially help students and advisors as they consider the activities that might improve students' odds of selection into competitive specialties.
Our specific aims were to identify the characteristics of medical students applying to the most competitive surgical specialties and examine the predictors of applicants who interviewed and successfully matched compared to applicants who interviewed and did not successfully match into a competitive residency specialty. Furthermore, we conducted a supplemental analysis to determine the effect of an away rotation at the applied program and geographical connection to the program, given below average USMLE Step scores. Because this was an exploratory study, we had no a priori hypotheses regarding which factors might predict matching success.

Database
We used data from the Texas STAR database. Data from four match cycles were used (2017 to 2020). Texas STAR was created by the University of Texas Southwestern Medical Center to help students and mentors navigate the match process more effectively and transparently [12]. Participating medical schools send their fourth-year students the Texas STAR survey on Match Day each year. In 2020, the database contained data from medical students at 115 United States medical schools. Data included self-reported board examination scores, honor society membership, research activities, publication information, volunteer experiences, among other factors, as well as which programs students applied to, interviewed at, and eventually matched [12]. The Texas STAR survey response rate averages 40% (unable to calculate for 2017, 47% in 2018, 38% in 2019, 47% in 2020). Several prior studies have utilized this database to evaluate applicant characteristics, expenditures, and success with matching into a surgical residency [13][14][15][16][17][18][19][20][21].

Selection criteria
We defined a surgical specialty as competitive if it was listed in the 2020 NRMP data as one of the most competitive specialties based on match rate with the highest percentage of U.S. medical school graduates [1]. The six most competitive specialties were (1) integrated plastic surgery, (2) otolaryngology, (3) neurological surgery, (4) thoracic surgery, (5) vascular surgery, and (6) orthopedic surgery. Students in the Texas STAR database were included if they participated in the survey from 2017 to 2020.

Outcome and predictor variables
Our primary outcome variable was matching into one of the six competitive specialties. Predictor variables included Alpha Omega Alpha honor status (AOA status), USMLE Step 1 score, USMLE Step 2 Clinical Knowledge (CK) score, other academic degrees, away rotation status (i.e., whether an away rotation was conducted at the program to which they applied), geographical connection, couples match status, number of honored clerkships (clerkship for which an A/Honors was awarded), honored specialty applying for (whether or not student received A/Honors for clerkship of specialty applied to), number of research experiences, number of peer-reviewed publications, number of abstracts/posters/presentations, number of leadership roles, and the number years off for research. Geographical connection was defined using the survey responses of students and may indicate the applicant has a hometown in the geographical region, attended college or research fellowship in the region, or conducted a visiting rotation in the region. The questions included on the Texas STAR survey are included in Appendix.

Statistical analysis
We calculated descriptive statistics for all variables and then conducted univariate analyses to examine differences between students matching into competitive specialties and students not matching into competitive specialties. We then calculated a multilevel logistic regression model to identify predictors of matching into a program. The multilevel logistic regression used data from each individual application, rather than data on an individual applicant. As such, each student applicant could be included multiple times in the regression analyses based on the number of programs they applied to. Multilevel modelling is commonly used in such situations, where there are multiple observations per participant (applications nested within individuals).
Categorical variables were analyzed using a chisquare test and continuous variables were analyzed using a Kruskal Wallis test. Statistical significance level was set at p < 0.05. Finally, we conducted a supplemental analysis using multilevel logistic regression to identify if completing an away rotation at the program or having a geographical connection to an institution increased odds of matching given above or below average USMLE Step 1 and Step 2 CK scores. This study was deemed exempt by the Institutional Review Board at the George Washington University School of Medicine and Health Sciences.

Basic student characteristics
Our sample included a total of 1,448 students who submitted 25,549 applications from 2017 to 2020. All students included in our analysis applied to a single specialty. A total of seven students were not offered an interview after applying. Basic student characteristics for those who applied to one or more of the six most competitive surgical specialties are listed in Table 1. On average, each student applied to 71 programs and interviewed at 18 programs. The majority of students in our sample (94%) had a Step 1 score greater than or equal to 230. Additionally, most students (92%) had a Step 2 score greater than or equal to 240. Of the students who applied to one or more of the six most competitive surgical specialties, less than half (43%) reported AOA status. Furthermore, we found that the average number of honored clerkships for students applying to one or more of the six most competitive surgical specialties was 4.4 (SD ±2.3). Although few students in the study had no peer-reviewed publications (N = 69), all of them had some type of research experience.

Predictors of successful matching
Academic and research variables were not associated with significantly increased odds of successfully matching into a particular program (Table 2). Students with a geographical connection to the program were just over one and a half times more likely to match compared to students without a geographical connection (adjusted OR, 1.65 [95% CI, 1.41 to 1.93], p < 0.001). Additionally, medical students who elected to do an away rotation at the program were 3.22 times more likely to match into a program compared to students who did not do an away rotation (adjusted OR, 3.22 [95% CI, 2.75 to 3.78], p < 0.001).

Away rotation status and geographical connection on matching
Students with a USMLE Step 1 score<230 and a USMLE Step 2 CK score<240 had a statistically significant increased odds of matching if they conducted an away rotation at the specific program (OR, 5.61 [95% CI, 3.17 to 9.92], p < 0.001). Additionally, students with Step 1 scores ≥ 230 and Step 2 CK scores ≥ 240 had increased odds of matching if they did an away rotation at the specific program (Table 3). Students with a geographical connection were at increased odds of matching; however, this result was only statistically significant for students with Step 1 scores ≥ 230 or Step 2 CK scores ≥ 240. Figure 1 represents the percentage of applications who had a successful match, stratified by specialty.

Match rate by specialty and year
Of the specialties analyzed in our study, most students interviewed at orthopaedic surgery (47%) or otolaryngology (24%) programs. Among the specialties included in our study, students interviewing at a neurological surgery program had the highest match rate after interviewing (88%), followed by otolaryngology (87%). On the other hand, students interviewing at a thoracic surgery program (55%) had the lowest match rate after interviewing. Furthermore, when comparing the rate of matching after interview for each year included in the study, the match rate did not differ significantly from year to year. Table 4 includes match rates for different specialties over the study period. αIncludes all students offered an interview; seven students were not offered an interview after applying. χThe number above the bars in the figure represent the match rate after interview.

Discussion
Providing medical students with evidence-informed information to guide their curricular and extracurricular activities is needed, as matching into competitive surgical residency programs continues to become more difficult. Additionally, such information can help guide residency career choices. In our study, we utilized the Texas STAR database to identify the factors related to successful matching for students applying to the six most competitive surgical specialties. Although the majority of students included in Table 3. Influence of away rotation and geographical connection on matching given USMLE step score.   our study had higher-than-average USMLE Step 1 and 2 scores, numerous research experiences, among other favorable selection criteria, these variables were not associated with an increased (odds) of matching into one of the most competitive surgical specialties. Additionally, we found that students with below average USMLE scores had increased odds of matching if they performed an away rotation at the program. In addition, an away rotation was more helpful for students with below average USMLE Step scores, as the odds ratio was higher for students with below average scores compared to above average scores. The NRMP's biennial Charting Outcomes of the Match research report is published to provide transparency to medical students regarding criteria for matching. The general characteristics of students applying to these competitive subspeciality programs are similar to those of the 2020 Charting Outcomes in the Match. This indicates that our sample may be generalizable to most allopathic medical students applying into these specialties, not just those at the institutions included in our sample. Individualized data on each application sent to every program that a student applied, which is included in Texas STAR, permits for further analysis of the factors influencing residency selection.
Various researchers have examined the selection criteria that contribute to successful matching into residency programs [3,7,10,22]. The Texas STAR database was used by Kremer et al. to examine the factors that increased the odds for matching into any medical specialty. Unlike that study, our study goal was to identify factors specific to students applying to competitive surgical specialties, since these criteria may differ. Additionally, we wanted to identify if geographical connections and away rotations could help students with below average USMLE Step scores. Kremer and colleagues examined data from 10,000 students included in the database to identify the odds of being offered an interview according to various academic, research, and extracurricular factors. The authors identified AOA status, Gold Humanism Honor Society, away rotation at a program, and geographical connection as increasing odds for an interview offer. In our study, we also found that geographical connection and away rotation status at a program increased odds of matching into a competitive surgical specialty. Findings from other studies on surgical subspecialties also align with some of our findings about specific samples of medical students applying to surgical subspecialties [23][24][25]. Specifically, USMLE scores, research experiences, AOA status, graduation from a top medical school, and having a surgical mentor were discussed as factors contributing to increased odds of matching.
Recent literature reinforces the importance of geographical location and away rotations on successful matching [22,26,27]. A study conducted by Gauer et al. found that matching into certain residency programs was significantly associated with USMLE Step 1 and USMLE Step 2 scores [8]. Indeed, the sample examined in our study was comprised of students who mostly had high USMLE Step scores. As such, range restriction was a factor and may partly explain why USMLE Step scores did not emerge as an important distinguishing factor in our sample of high-achieving medical students. In other words, students applying to the surgical specialties included in our study typically apply with competitive board scores, as this is integral to higher match success; thus, mentors and advisors likely encourage students to consider their USMLE Step performance when counseling them on applications. Nonetheless, we found that conducting an away rotation at a program was statistically significantly associated with increased odds of matching, despite USMLE scores. Higher USMLE Step scores, better academic performance, and AOA status are, intuitively, factors that would increase the odds of an individual matching into a particular specialty as they are indications of outstanding academic performance. For competitive surgical specialties, we suspect there is less variation in these criteria. Most students applying to the specialties included in this study have multiple research experiences and above average USMLE Step scores. Therefore, more subjective measures, including experiences and performance on an away rotation, potential networking on an away rotation, letters of recommendation, and interview performance may weigh more heavily for this specific population. Although we cannot evaluate the qualitative aspects of the selection process, including interview and personal statements, this is an avenue of future research.
Away rotations may provide medical students a better understanding of their desired specialty, and these rotations give students the opportunity to establish relationships with mentors who may aid in their success in a particular field. Fereydooni and colleagues examined medical students' motivations for pursuing a career in integrated vascular surgery and the logistics of away rotations. The authors found that 83% of students matching into an integrated vascular surgery residency completed an away rotation. This study highlighted the importance of away rotations in permitting students an opportunity to assess the factors that foster their choice to pursue residency at a particular program [28]. Additionally, Baldwin et al. retrospectively reviewed medical student application data for orthopaedic surgery and found that the number of away rotations performed by a student increased odds of a match success [27]. Additionally, an away rotation may allow for the development of a meaningful relationship with a mentor, that may lead to a letter of recommendation or involvement in research, which may further increase an applicant's chances to match into a particular program.
Costs associated with residency applications and away rotations in surgical subspecialities have been investigated in the literature and evidence shows that the costs for applying to certain surgical subspecialties is higher than other subspecialties [13,16,20,29,30]. Specifically, Gordon et al. used the Texas STAR database to compare the costs of students applying to competitive surgical residencies, and the authors found application costs to vary depending on subspeciality and geographic region [20]. Tese findings highlights the need to develop strategies to reduce variation in the costs of residency applications and limit costs overall. Away rotations may represent substantial financial burden for students applying to certain specialties, presenting an opportunity to develop strategies to combat the high costs associated with residency applications. For example, research indicates that some students may pay as much as $2,500 to perform away rotations [28]. Given the potential advantages of away rotations, as highlighted in the current study, it is important to ensure that these experiences are available to all students, regardless of their financial status or resources available at their home institution. For example, strategies may be adopted by host and welcoming institutions to provide low-resource students the opportunity to attend an away rotation or reap similar benefits. Such strategies may include stipends, food allowances, complementary housing, among other methods [26,28,31].
In addition, the COVID-19 pandemic led to the development of virtual electives for students who could not rotate in-person. Best practices for virtual experiences could be shared to maximize their benefits for learners. Furthermore, some students do not perform well in a standardized testing environment or have structural barriers to testing performance, such as less access to test preparatory materials, and an away rotation may present them with the opportunity to showcase their knowledge in a practical way. Thus, institutions who require a particular score before even offering an invitation to do an away rotation should consider assessing applicants in a more holistic manner. Finally, the development of a national policy limiting the number of away rotations that students can perform could promote equity, as well as address overall costs.
Geographical connections to a particular program or region may communicate meaningful information to a residency committee in terms of likelihood of a student's sincere interest in the program. Such data may be communicated through permanent address, school address, personal statements, interview, or through leadership and volunteer experiences. Recently, the Electronic Residency Application Service (ERAS) introduced geographic preference capabilities within their supplemental application during 2021-2022 and 2022-2023 residency application cycles [32]. Additional insights from their data will be helpful in understanding the importance of communication around applicant geographic preferences.
Our study has limitations. The Texas STAR database contains self-reported fourth-year student data that is completed on Match Day or later each year, exposing the dataset to potential recall bias. Nonetheless, students have access to their application data after submission. However, the mean student board scores in our study match the NRMP's Charting Outcomes in the Match scores, likely indicating a representative sample. Importantly, this database does not contain data regarding interviews, letters of recommendation, class ranking, personal statements, and other factors that may predict successful matching. Future research might consider collecting data from program directors on interview performance and the quality of letters of recommendation at particular programs to determine the influence of interviews. Nonetheless, we included data from over 1,448 students and 115 medical schools (out of a total of 155 total accredited MDgranting institutions in 2019) [33]. Finally, because these data come from a cross-sectional survey, we cannot make causal inferences regarding the relationships observed in this study. We acknowledge that successful matching and match rates for specialties are influenced by many factors, including number of positions available, interview performance, and other factors not included in this study. Numerous metrics can be used to define a surgical specialty as competitive, including size of the applicant cohort compared to the number of positions available and number of ranks per applicant needed to match in a given specialty. Nonetheless, we decided to utilize the match rates listed in the NRMP report to define a surgical residency as competitive. Future studies may focus on identifying which metric is most indicative of competitiveness. Additionally, the match algorithm is driven by applicant preference and this influences where a student ultimately matches. Although the Texas STAR database includes data from students in numerous medical schools, the absence of data from all accredited MD schools may influence differences in match rates observed in our study and NRMP's Charting Outcomes data. Multilevel logistic regression did not control for specific specialty, and this may influence the results as certain specialties vary in regards to the value of specific applicant characteristics during ranking.

Conclusion
In this analysis, examining predictors of matching into competitive surgical specialties, we found that away rotations completed at the program to which a student applies to and geographical connections are integral components of the selection process. Away rotations provide medical students with immense opportunities to network, get involved with specialtyspecific research, and familiarize themselves with the specialty, among other benefits. Given the financial implications associated with away rotations, institutions across the country should work to ensure all students have access to these benefits, regardless of socioeconomic status or their ability to perform an away rotation. Future work should aim to gather more comprehensive evidence regarding the residency selection process to provide further transparency for students as they prepare their residency applications to programs across the country. Additionally, these studies may identify trends in the applications submitted by students to determine the effect of various medical education policies on the residency application process.

Highlights
(1) Students with geographical connection to an institution or students who performed an away rotation at an institution are more likely to match into competitive surgical specialties. (2) Students with below average board scores had increased odds of matching into a competitive surgical specialty if they completed an away rotation at the applied program. (3) Away rotations and geographical connections to an institution may contribute more than academic criteria for selection into a competitive surgical residency after an interview, as academic criteria may not be distinguishing in this self-selected applicant pool.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
The author(s) reported that there is no funding associated with the work featured in this article.